From the Support Vector Machine to the Bounded Constraint Machine

The Support Vector Machine (SVM) has been successfully applied for classification problems in many different fields. It was originally proposed using the idea of searching for the maximum separation hyperplane. In this article, in contrast to the criterion of maximum separation, we explore alternative searching criteria which result in the new method, the Bounded Constraint Machine (BCM). Properties and performance of the BCM are explored. To connect the BCM with the SVM, we investigate the Balancing Support Vector Machine (BSVM), which can be viewed as a bridge from the SVM to the BCM. The BCM is shown to be an extreme case of the BSVM. Theoretical properties such as Fisher consistency and asymptotic distributions for coefficients are derived, and the entire solution path of the BSVM is developed. Our numerical results demonstrate how the BSVM and the BCM work compared to the SVM.